Benchmarking the Impact of Faulty Drivers: Application to the Linux Kernel

Dependability concerns, encompassing robustness assessment, are essential questions to answer before a developer can make the decision whether to integrate off-the-shelf (OTS) components into a dependable system. Here, and in what follows, robustness is understood as the degree to which a system operates correctly in the presence of exceptional inputs or stressful environmental conditions, in compliance with the generic definition (dependability with respect to external faults) given in [Avizienis et al. 2004]. From a cost-effectiveness viewpoint, operating systems and kernels are privileged OTS components as candidates for integration into a system. However, integrators are often reluctant to make such a move without obtaining a deeper knowledge and understanding about such a component beyond functional issues, in particular with respect to its failure modes and its behavior in the presence of faults. Due to the opacity that is often attached to the commercial offer and to the difficulty and significant cost associated with the availability of the source code, the Open Source option, for which access to the source code is granted, is progressively making its way as an attractive and promising alternative. Also, results of many studies have shown that Open Source solutions did not exhibit significantly more critical failure modes and in some cases they were even found to demonstrate behaviors superior to commercial options [Koopman and DeVale 1999, Arlat et al. 2002, Marsden et al. 2002, Vieira & Madeira 2003]; see also Chapter 5 for the latter. In this chapter, we will simply denote such components (either commercial or Open Source) as OTS components. In the past years, several experimental studies have addressed this important issue from different perspectives [Koopman and DeVale 1999 (see also Chapter 11), Arlat et al. 2002, Madeira et al. 2002]. This has also led to the proposal of tentative dependability benchmarking approaches, aimed at characterizing the robustness of computer systems